On the Accurate Determination of Search Directions for Simple Differentiate Penalty Functions

We shall assume that each problem function has sufficiently many continuous derivatives for any implicit assumptions that we make to hold. One of the most successful tools for solving problems of the form NLP and SIP is the penalty function. A penalty-function approach replaces the relevant problem by a suitably weighted combination of the objective function f(x) and functions representing violations of constraints. This weighted combination is known as a penalty function. The unconstrained penalty function is normally minimized for a particular choice of the weighting and the weighting subsequently adjusted. The rationale behind such methods is the existence of powerful theoretical results which indicate how the weighting should be adjusted and when the minimizer of the penalty function is likely to converge to that of the relevant original problem (see Fiacco & McCormick, 1968; Pietrzykowski, 1970). Early penalty functions for NLP include the quadratic loss function (form mixed equality and inequality constraints) and the barrier functions (for inequality constraints):